1. Field of the Invention
The present invention relates to the field of computer graphics and, in particular, to a system and method for art-directable retargeting for streaming video.
2. Description of the Related Art
Traditionally, motion pictures and/or videos have been configured to be played back on a specific target platform, such as in a cinema screen or on a television. Examples include movies displayed in a movie theater or digital broadcast content displayed on television, where the movies may be film movies or digital cinema. In recent years, however, there has been an increasing demand for displaying video content on devices with considerably differing display formats, such the display formats used on mobile phones or MP3 players.
In order for the video content to be properly played back on these devices, the source video content must be “retargeted” to a target video format. Retargeting of video content includes performing one or more video processing operations so that the video content is better suited for the target platform. Conventional techniques for video retargeting implement a naive linear downscaling, where the source content is linearly scaled (both vertically and horizontally) to fit within the frame size of the target platform.
However, one problem with naive linear downscaling for video playback platforms such as those used on mobile phones or MP3 players is that the resultant video does not provide for a comfortable viewing experience since the proportions of the objects in the video may appear unnatural. Similar issues occur for DVD players and/or next generation free-form displays.
To overcome the problems with retargeting video using linear downscaling, a variety of techniques have been investigated that remove “unimportant” content by cropping and/or panning. Cropping techniques include trimming the width and/or height of a source video to achieve a target video with a lesser width and/or height to fit within a target video format. Panning techniques are similar to cropping techniques, where a portion of the source video is selected that matches the frame size of the target video format.
A different class of approaches removes unimportant content from the interior of the images or video. These techniques compute a manifold seam through the image data in order to remove insignificant pixels. While these approaches have shown very promising results, they are still subject to significant conceptual limitations. For example, since the seam removes exactly one pixel per scanline along the resized axis, large scale changes inevitably result in seams cutting through feature regions. In addition, the removal of certain pixels can cause visible discontinuities or aliasing artifacts.
Each of the above techniques suffers from further limitations when used for performing video retargeting, as opposed to image retargeting. In video retargeting, a number of additional issues are raised, such as temporal coherence of the warp from source format to target format. However, conventional video retargeting approaches are not capable of uniform scaling important image content.
As the foregoing illustrates, there is a need in the art for an improved technique for image and video retargeting that overcomes the limitations of conventional approaches.